Down-line transcription system using automatic tracking and revenue collection

ABSTRACT

A transcription network having linked computer terminals for a court reporter and for examining, defending, and associate attorneys is disclosed. Real-time transcription text is communicated from the reporter&#39;s terminal through the network. Secure message interchange between the examining attorneys and between defending attorneys is provided along independent communication links and using encryption. Off-site terminals for clients, experts, or other associate attorneys may be linked to the network. Communication on the network is hierarchically controlled so as not to bombard the examining attorney, and is managed by a protocol for updating disconnected and non-listening terminals. The terminals provide software routines for automatically delivering proposed objections to the form of a question or answer. Stenographic keystrokes that cannot be directly transcribed are converted to a phoneme form for down-line readability. A synchronization scheme provides for automatic play-back of audio and video recordings corresponding to the transcribed text of a specific question or answer.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.09/422,202, filed Oct. 19, 1999, now U.S. Pat. No. 7,631,343, which:

1) is a continuation-in-part of U.S. patent application Ser. No.09/185,114, filed Nov. 3, 1998, now U.S. Pat. No. 6,023,675, issued Feb.8, 2000, which is a continuation of U.S. patent application Ser. No.09/057,650, filed Apr. 9, 1998, now U.S. Pat. No. 5,970,141, issued Oct.19, 1999, which is a divisional of U.S. patent application Ser. No.08/818,402, filed Mar. 12, 1997, now U.S. Pat. No. 5,949,952, issuedSep. 7, 1999, which is a continuation-in-part of U.S. Pat. Ser. No.08/339,771, filed Nov. 15, 1994, now U.S. Pat. No. 5,740,245, issuedApr. 14, 1998, which is a continuation of U.S. patent application Ser.No. 08/066,948, filed May 24, 1993, now U.S. Pat. No. 5,878,186, issuedMar. 2, 1999, which is a continuation of U.S. patent application Ser.No. 08/036,488, filed Mar. 24, 1993, now U.S. Pat. No. 5,369,704, issuedNov. 29, 1994; and

2) claims the benefit of U.S. Provisional Patent Application Ser. No.60/140,760, filed Jun. 24, 1999.

The complete contents of the above referenced U.S. patent applications,U.S. Pat., and U.S. Provisional Patent Application are herebyincorporated by reference in their entirety.

BACKGROUND OF THE INVENTION

This invention relates to a transcription system used by courtreporters; and, more particularly, it relates to a method and apparatusincorporating an automatic transcription system for providing real-timeuse and manipulation of transcribed testimony by attorneys, judges,court reporters, witnesses and clients.

As is well known, legal proceedings such as a deposition or trialinvolve the participation of, among others, an examining attorney whoasks questions and a witness who must answer (“testify”) while underoath. These answers (“testimony”) are recorded by the court reporter,along with the associated questions and related conversation, using astenographic recorder. A stenographic recorder is a machine whichprovides a set of keys which are stroked by the court reporter invarious combinations and sequences to represent a spoken word. Toprovide a backup to the key-strokes, court reporters use a tape recorderto record tile entire proceeding.

Older versions of the stenographic recorder only record the courtreporter's key-strokes much the way a computer printer does, creating apaper tape containing the printed key-strokes as a record of the legalproceeding. To the untrained eye, the printed key-strokes areincomprehensible. After the legal proceeding ends, the court reporterreads the paper tape and manually transcribes each printed key-strokeback into the words that were spoken, creating a readable transcript ofthe testimony. This manual process is herein called “manual,post-processed transcription”.

Newer versions of stenographic recorders have been developed and linkedto computer aided transcription (“CAT”) systems to help automate themanual transcription process. Instead of solely using paper taperecording methods, the stenographic recorder also electronically storeskey-strokes in built-in memory or on disk. After using such a newerrecorder, the court reporter returns to his office and transfers theelectronically stored key-strokes to his CAT system for transcription.Although the bulk of electronically stored key-strokes may be translatedautomatically, the court reporter must still work interactively with theCAT system to translate those key-strokes which the CAT system could notrecognize. This nearly automated transcription process is herein called“automatic, post-processed transcription”.

The most recent versions of the stenographic recorder also accommodate“automatic, parallel transcription”, by electronically sending eachelectronically registered key-stroke via a telephone line to a remoteCAT system during the legal proceeding. This system requires two courtreporters working in parallel: one using the stenographic recorder torecord the testimony; and the other, who is familiar with the firstreporter's key-stroking style, interacting with the CAT system to carryout the transcription of each key-stroke received. Working in parallel,the court reporters might produce a readable transcript within a fewhours after the proceeding has ended.

For all of the transcription systems mentioned above, court reportersoften use the tape recorder to aid in translating misheard words anduntranslated key-strokes. To locate the corresponding audio on arecorded tape often wastes a great deal of the court reporter's time.

Automatic, parallel transcription systems have also been unsuccessfullyused in a configuration to assist a deaf witness in legal proceedings.Taking the place of the second, parallel court reporter in theaforementioned normal scenario, the deaf witness is placed in front ofthe CAT system. The CAT system translates and displays the bulk of thekey-strokes and sequences in real-time so that the deaf person can readthe questions asked. One major problem found in this configuration isthat the untranslated key-strokes are displayed in an undecipherableform to all but a court reporter. A relatively large number of thequestions asked become unintelligible. Furthermore, the fact that thewitness is deaf complicates this matter to a point of failure. For everyunintelligible question encountered, the court reporter must stop usingthe stenographic recorder, read the computer screen and then write downthe translation by hand on a piece of paper for the witness. As anadditional problem the witness's own words annoyingly echo back to thecomputer screen within a short delay after they are spoken, making itvery difficult for the witness to testify without distraction.

The development of the automatic, post-processed and paralleltranscription system was driven solely by the court reporter's need toaccelerate the transcription process. Using these systems, the courtreporter operates more efficiently, handles a larger volume of work and,as a result, reaps greater profit. Using any of these transcriptionsystems, however, a readable transcript is never received by anattorney, judge or witness until some time after the immediate legalproceeding has ended.

There are a variety of problems facing attorneys, judges and witnessesrelating to the development of testimony taken in a legal proceeding.The focal point of most of these problems involves the lack of areal-time, readable record of a witness's testimony while that witnessis undergoing examination. Reliance must be based upon either theminimal assistance the court reporter might offer or notes taken whichoften prove to be burdensome, confusing and untrustworthy. Furtherdetails of these problems become apparent when evaluating the type oflegal proceeding involved.

A deposition proceeding often involves the participation of a witness,court reporter, examining attorney, associate examining attorney,defending attorney and associate defending attorney. Generally, theexamining attorney asks questions to which the witness responds withanswers. The associate examining attorney assists the examining attorneyin conducting this inquiry. The defending attorney, with assistance fromthe associate defending attorney, evaluates the questions being askedand raises objections when legally improper questions are detected. If alegally improper question occurs but is not detected, the correspondinganswer may be used to the detriment of the defending attorney's case.During the entire proceeding, the court reporter records all of thequestions, answers, objections and discussions held during thedeposition.

In parallel with the court reporter's recording of the proceeding, allattorneys rapidly take notes in which they attempt to summarize whatthey believe to be the important factors of what they believe they heardthe witness say. Although these notes provide a necessary, workingrecord to aid in conducting the deposition, they are often incomplete,mischaracterize or, even worse, completely misconstrue a witness'sanswers. Relying on such notes often proves to be detrimental.

If an attorney does not understand an answer, he often asks the courtreporter to read the answer back from the paper tape record. Theexamining or defending attorney may also ask for the last answer to beread back if the answer was so lengthy that notes were not taken fastenough to keep up with the witness. Similarly, either attorney may askthe court reporter to read back the last question for a confusedwitness. When so instructed, the court reporter stops recording, picksup the paper tape output from the stenographic recorder, searches forthe portion of the record at issue, and reads the stenographickey-strokes. This reading is often broken with misplaced delays inpronunciation between phonemes and words making it difficult tounderstand. Furthermore, because searching for a portion of the recordin a series of printed key-strokes proves to be very difficult and timeconsuming, attorneys are forced to limit requests to have only the mostrecent Q & A's read back.

More specifically, the examining attorney takes notes in order to helpformulate questions during a deposition. These notes are referencedparticularly as the witness reveals additional evidence about the samesubject. If the deposition runs for several days, the examining attorneyoften reviews each day's notes during the evening in his hotel room inpreparation for the next day of deposition. Reviewing cryptic notestaken many hours earlier often causes further misunderstandings whichmay not be detected until a readable transcript is received. Operatingfrom unreliable notes, the examining attorney might incorrectly: 1)proceed with irrelevant lines of questioning; 2) reopen resolved,important issues—tipping-off the defending attorneys; or 3) abandonunresolved, important lines of questioning.

A major goal of the examining attorney is to establish the proper formof Q & A's which relate to important issues in a case. In furtheringthis goal, the examining attorney often attempts to embody one or moreof the witness's prior answers as part of a new, summarizing question.Because this new question is often based upon unreliable notes or poormemory, the new question mischaracterizes the prior answers. Inresponse, the defending attorney may correctly object if he detects themischaracterization from his potentially unreliable notes and poormemory. The objection forces the examining attorney to consider eitherasking the court reporter to try to find and read back the actual Q &A's at issue or re-asking the entire line of questioning again. Moreimportantly, this objection warns the witness to carefully scrutinizeeach question relating to this subject before answering. As a result, abattle usually results over superfluous terms and phrases, and obtainingproper form is prevented. Furthermore, even if the examining attorneybelieves proper form has been achieved, he is often reluctant to verifyhis belief by having the Q & A read back because it serves to alert thedefending attorneys that important information may have come to light.Thus, improper form on critical issues often results.

Leeching away his time for, and therefore the reliability of, his notetaking, the examining attorney is also faced with a multitude of othertasks such as: 1) analyzing the response of the witness; 2) formulatinga follow up question in view of the witnesses response; 3) analyzing thedefending attorneys objections to the pending question; 4) evaluatingthe demeanor of the witness upon each Q & A; 5) reviewing an outline forthe next preplanned question or line of questioning; and 6) writingfurther notes regarding future lines of questioning to be asked orreviewed that evening at the hotel.

To relieve him of some of the burden, the examining attorney oftenbrings an associate examining attorney with him. The associate doesseveral things including: 1) taking notes to fill gaps in the examiningattorneys notes at a break or later that evening at the hotel when thetwo attorneys plan the next day's questions; 2) ordering and findingdocumentary exhibits for the examining attorney; 3) writing messages onsmall pieces of paper (usually POST-IT brand notes) and passing them tothe examining attorney; 4) monitoring the passed messages so as towithdraw, supplement or replace them when necessary; and 5) keepingtrack of the documentary exhibits entered into evidence.

While relieving some of the burden upon the examining attorney, theassociate examining attorney creates additional burdens. For example, insuggesting a follow-up question, the associate must rapidly compose abrief message and immediately obtain the attention of the examiningattorney before the examining attorney changes the line of questioning.As a result, the examining attorney sometimes forgets his own line ofquestioning. Moreover, these messages tend to be illegible and too brieffor the examining attorney to understand. Without supplementation, thesemessages serve no purpose but to distract.

To suggest new lines of questioning, the associate may casually draft arather complete message intending for the examining attorney to read themessage at the examining attorney's leisure. However, the mere act ofpassing the note tends to distract the attention of the examiningattorney.

The associate attorney may also pass messages to help counter thedefending attorneys' objections or pointing out the witness's demeanorin response to a question when the examining attorney is lookingelsewhere. Again, this distracts the examining attorney.

Upon receiving a message, the examining attorney must decide when toread it, and, once read, must decide whether and when to use it. Ifprior to reading the message, the examining attorney asks the questioncontained therein, the message becomes no more than a distraction. Ifthe examining attorney determines that the message contains a goodquestion, the examining attorney either asks that question immediatelyand discards the note or saves the note for a later time. At times, anattorney may have ten to fifteen notes in front of him that he must dealwith. This is particularly the case where the associate knows more factsabout the witness than does the examining attorney, or where theassociate attorney is the more senior or more experienced of the two.

As previously stated, the defending attorney must also take summarizingnotes regarding both the Q & A's in the deposition. From these notes,the defending attorney attempts to deduce upcoming lines of questioning,specific details of the examining attorney's positions, strengths orweaknesses associated with those positions and objections such as “askedand answered” or “mischaracterizing the testimony”. These notes alsoremind the defending attorney to discuss incorrect statements made bythe witness during a break so that the record can be corrected when thedeposition resumes.

Among a variety of other duties also detracting from his note takingability, the defending attorney must object whenever the question isinappropriate and state the reasons for his objections. Not only dothese objections address evidentiary concerns, but some attorneys usethe objection to warn the witness to scrutinize critical questions posedby the examining attorney as previously described. Furthermore, all suchobjections must be seasonable, i.e., before the witness answers apending objectionable question. Note taking tends to detract from thedefending attorney's ability to seasonably object and vice versa.

To help alleviate this problem, the defending attorney often bringsalong an associate defending attorney. Much like the associate examiningattorney, the associate defending attorney takes notes and passesmessages to the defending attorney. If the note involves making anobjection, the defending attorney will rarely be able to make itseasonably. The messages just cannot be drafted, passed and read fastenough. The notes taken do however help fill the gap found in thedefending attorney's notes who can then focus on his other duties. Inaddition, the description of many of the other problems encountered bythe examining attorneys are equally applicable here.

During a trial proceeding, both the plaintiff's attorneys anddefendant's attorneys take turns assuming the role and duties of theexamining and defending attorneys as previously described in thedeposition proceedings. In many cases, instead of a single associateattorney, many associate attorneys may participate. The examiningattorney at trial is called the “first chair”. The most senior associateattorney, if at least one, is called the “second chair” and so on. Theremay be several “chairs” representing a single client.

The problems described previously relating to a deposition directly, andperhaps more severely, apply to a trial proceeding. This is due in partto the increased number of attorneys involved. Additionally, severaldifferent problems associated specifically with the trial proceedingalso exist. For example, because the first chair is often on his feetand not near the other chairs on his team, passing messages or notetaking proves to be very difficult if not impossible. This communicationgap manifests other problems when the first chair attempts to obtaintrial exhibits or other supporting materials. Scrambling to draft andpass a message to, or to reach whispering range of, a lower number chairoften occurs. Other chairs that could possibly know of the location ofthe requested material are often oblivious to the request.

Another communication gap exists between the attorneys at trial andthose that cannot attend. Only those attorneys present and listening canrespond at the proceeding. Non-attendees who might otherwise be able toassist have no means to evaluate the trial or communicate with theattending attorneys.

On the defending attorney's side, the nomenclature using the firstchair, second chair and so on is also used. The problems applicable tothe defending attorneys during a deposition are also applicable andcompounded at trial because of the enhanced significance of thedefending attorney's duties beyond those of detailed note taking. As aresult, much of the note taking must be left to other chairs.

A judge must monitor, evaluate, and moderate the procedures andsubstance of the trial proceedings. Each question or line ofquestioning, answer and exhibit must be carefully analyzed for content,evidentiary form and relevance so that rulings on related objections canbe immediately made. If a judge is distracted, he is faced with problemspreviously enumerated with having to have the court reporter read backfrom the record. The judge also takes notes relating to various Q & A'sor statements made during trial, particularly when ruling on issues oflaw or fact from the bench. Due to the multitude of duties facing thejudge, the judge's note taking is also subject to many of the otherpreviously mentioned problems.

Currently, facing the foregoing problems are over thirty thousand courtreporters and hundreds of thousands of attorneys and judges in theUnited States alone. Hence, it would be highly desirable to solve theforegoing variety of problems enumerated above facing attorneys, judges,court reporters and witnesses in conducting legal proceedings such as adeposition or trial by using and manipulating testimony generated inreal-time by an automatic transcription system.

It is therefore an object of the present invention to provide a methodand apparatus which will aid the examining attorney's use andmanipulation of testimony generated in real-time by an automatictranscription system.

It is another object of the present invention to provide a method andapparatus which aids associate examining attorneys in the use andmanipulation of testimony generated in real-time by an automatictranscription system to assist the examining attorney.

It is yet another object of the present invention to provide a methodand apparatus which aids the defending attorney in the use andmanipulation of testimony generated in real-time by an automatictranscription system.

It is another object of the present invention to provide a method andapparatus which will aid the associate defending attorney's use andmanipulation of testimony generated in real-time by an automatictranscription system in assisting the defending attorney.

It is therefore an object of the present invention to provide a methodand apparatus which will aid the examining first chair's use andmanipulation of testimony generated in real-time by an automatictranscription system.

It is another object of the present invention to provide a method andapparatus which will aid the examining second chair's use andmanipulation of testimony generated in real-time by an automatictranscription system in assisting the examining first chair.

It is another object of the present invention to provide a method andapparatus which will aid all examining higher chairs' use andmanipulation of testimony generated in real-time by an automatictranscription system in assisting the examining second chair.

It is yet another object of the present invention to provide a methodand apparatus which will aid the defending first chair's use andmanipulation of testimony generated in real-time by an automatictranscription system.

It is another object of the present invention to provide a method andapparatus which will aid the defending second chairs' use andmanipulation of testimony generated in real-time by an automatictranscription system in assisting the defending first chair.

It is another object of the present invention to provide a method andapparatus which will aid all defending higher chairs' use andmanipulation of testimony generated in real-time by an automatictranscription system in assisting the defending second chair.

It is a further object of the present invention to provide a method andapparatus which will aid the judge's use and manipulation of testimonygenerated in real-time by an automatic transcription system.

SUMMARY OF THE INVENTION

These and other objects of the present invention are achieved in atranscription network having a reporter's terminal, an attorney'sterminal, and an associate's terminal with a communication linktherebetween. Specifically, the attorney's terminal and associate'sterminal receive text signals representative of spoken words from thereporter's terminal via the communication link, and display the text.The associate's terminal also provides a means for generating andsending messages via the communication link to the attorney's terminal.

Other objects are also achieved in a transcription system having aterminal, for transcribing signals representative of spoken words intocorresponding alphabetic and numeric text, which is linked to a recorderfor recording spoken words. A means which operates across the linkcreates associations between recorded spoken words and correspondingalphabetic and numeric text.

Objects are also achieved in a reporting system having a transcribernode which operates on coded representations of spoken words and havingcross-reference and phoneme libraries. Specifically, the reportingsystem uses the cross-reference library to identify; associationsbetween coded representations of spoken words and correspondingalphabetic and numeric text. Where associations cannot be identified,the reporting system then uses a phoneme library for associating spokenwords with pronounceable text.

Objects are also achieved in a reporting system operable on codedrepresentations of spoken words. Particularly, the reporting systemcomprises a transcriber which cross-references the coded representationsof spoken words so as to identify corresponding alphabetic and numerictext. The identified text is then analyzed and, if appropriate,objections to the spoken words are provided.

Objects are also achieved in a transcription network comprising aplurality of terminals each having a display screen which iselectrically controllable for visually displaying alphabetic and numerictext. A hierarchical communication link controls the communication flowbetween the plurality terminals. Each of the terminals send and receivemessages via said hierarchical communication link, which provides forselective relaying of messages between said plurality of terminals.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of an embodiment of an overall systemconfiguration according to the present invention.

FIG. 2 is a schematic block diagram of an embodiment of the stenographicrecorder used in the present invention.

FIG. 3 is a detailed view of the interconnection of the stenographicrecorder and CAT system.

FIG. 4 a is a diagram representing the association of data fields into arecord which is used as a basis for the overall data structure fortranscription according to the present invention.

FIG. 4 b is a detailed diagram representing the overall data structureused by the CAT system in transcription.

FIG. 5 a is a diagram representing the phoneme library used in thepresent invention wherein substitutions of readable phoneme text isfound for untranslated key-strokes.

FIG. 5 b is an exemplary diagram of a foreign language table which isused to cross-reference text of a first language that corresponds totranslated words of a first language.

FIG. 6 a is a perspective view of the examining attorney terminal andits interconnection to the CAT system and stenographic recorder.

FIG. 6 b is a perspective view of the examining attorney terminal'swhich provides further detail for both the display and keyboard.

FIG. 7 a is a perspective view of the examining attorney terminaldisplaying the outline mode of the present invention which provides textediting of a case law outline indexing a set of preset messages.

FIG. 7 b is a perspective view of the examining attorney terminaldisplaying a tree-structured outline providing hierarchical display ofthe categories of law to be investigated during a deposition.

FIG. 8 is a perspective view of the associate examining attorneyterminal which provides details in both the display and keyboard and itsinterconnection to the examining attorney terminal, CAT system andstenographic recorder.

FIG. 9 is a perspective view illustrating the use of audio or videotracking in the present invention.

FIG. 10 is a perspective view of the wireless and hardwiredinterconnection of an alternate embodiment of the transcription systemin a court room setting.

FIG. 11 is a perspective view of another embodiment of the presentinvention which cooperatively utilizes stenographic and voicetranscription.

FIG. 12 is a diagram illustrating two alternate variations of thepresent invention where the court reporter terminal is not linked to thetranscription network.

FIG. 13 is a perspective view of another embodiment of the presentinvention where, instead of using a keyboard, the attorney terminals 15and 17 utilize pen-based or touch-sensitive screen input.

FIG. 14 illustrates a flow-chart describing the communication protocolused by the attorney terminals for communicating with the reporterterminal (or CAT system) 21.

FIG. 15 illustrates an alternate flow-chart describing the reporterterminal's functionality in carrying out the Q&A communication protocol.

DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 1 is a perspective view of an embodiment of an overalltranscription system configuration according to the present invention. Astenographic recorder 11 is used by a court reporter at a deposition,hearing or other transcription matter, to record digital coded signalsrepresentative of verbal communications as they occur in real-time.Using a communication link 19, the recorder 11 transfers therepresentative signals to a computer aided transcription (“CAT”) system13, a computer terminal used by the court reporter, for transcriptioninto alphabetic and numeric text corresponding to the actual wordsspoken.

The CAT system 13 communicates the alphabetic and numeric text itgenerates along two independent communication links 20 and 21 to anexamining attorney's terminal 15 and a defending attorney's terminal 16,respectively. The examining attorney's terminal 15 relays the text alonga communication link 23 to an associate examining attorney's terminal17. Likewise, the defending attorney's terminal 16 relays thecommunicated alphabetic and numeric text representing the spoken wordsalong a communication link 22 to an associate examining attorney'sterminal 18. Upon receipt of the communication, the attorney's terminals15, 16, 17 and 18 display the alphabetic an numeric text and provide avariety of tools for reviewing and evaluating what has been received.

Concurrent with receiving and displaying representative text, theexamining terminals 15 and 17 provide a vehicle for the examining andassociate examining attorneys to exchange messages. The message exchangeoccurs through the communication link 23. Similarly, message exchangesbetween the defending terminals 16 and 18 are provided for via thecommunication link 22.

As illustrated, the CAT system 13 may be positioned for easy access on aconference room table 25 in front of the court reporter. Also on thetable 25, the terminals 15, 17, 16 and 18 may be found positioned infront of an examining attorney, associate examining attorney, defendingattorney and associate defending attorney, respectively. Each of theterminals 15, 16, 17 and 18 and the CAT system 13 (the reporter'sterminal) all include keyboards 27 and a screen 29. Each screen is flator planar so as to prevent viewing by the opposing attorneys (andwitness) who typically sit on either side of the table, as shown.

Although preferred, neither the keyboard nor the screen are necessaryfor the CAT system 13. In fact, the terminal itself, i.e., thefunctionality thereof, might exist within other nodes on thetranscription network. For example, the functionality of CAT system 13might be fully distributed between the recorder 11 and the examiningattorney's terminal 15. The functionality might also be located at someremote, off-site location. Similarly, the present invention contemplatesmany situations where terminals are not available for each attorneypresent, such as where only the examining side uses terminals or visaversa, or where either side utilizes a single terminal. Situations mayarise where no attorneys possess a terminal.

The use of additional terminals is also contemplated by the presentinvention. Although not shown in FIG. 1, a sixth terminal might beused: 1) for the witness who sits to the left of the defending attorneyon the defending attorney's right side; 2) off-site for a magistrate orjudge; 3) off-site for a client or expert witness; or 4) off or on sitefor additional attorneys on the case. These configurations and theiradvantages and features will be discussed in detailed below.

FIG. 2 is a schematic block diagram of an embodiment of the stenographicrecorder 11 used to generate digitally coded stenographic signalsrepresenting the voice transcript. Recorder 11 includes a keyboard 31which is actuated by the court reporter in response to the spoken word.The reporter strokes the keyboard by actuating either one keyindividually or several keys simultaneously to represent phoneticallythe spoken word or some phonetic portion thereof. Such a key-strokecauses print actuator 33 to print a single line of type representingthat key-stroke onto a paper tape 35. Tape 35 is thereafterincrementally scrolled out in preparation for the next key-stroke.

During the deposition, the court reporter may be asked to read backtestimony. The reporter does so by sequentially viewing each line ofphonetic characters typed onto paper tape 35, mentally translating asingle or several lines of characters into words, and then speaking outthe phonetic translation. An attorney, judge or witness typically cannotread the phonetic characters typed onto tape 35.

In addition to being printed on paper tape 35, each key-stroke isconverted to a digital code which is stored in a random access memory(RAM) 37 and also transmitted by a communication port 39 to CAT system13 via a serial output bus 41 and communication link 19 (FIG. 1). Ifprint actuator 33 runs out of tape 35, stenographic recorder 11 willstill store and transmit the digital key-stroke codes. Similarly, ifcommunication link 19 is disconnected, storage and printing will stilloccur.

More specifically, a keyboard scanner 43 monitors keyboard 31 viascanning bus 45. Upon each stroking of keyboard 31, keyboard scanner 35responds by generating a digital key-stroke code representing all of thekeys pressed in that stroke. Next, keyboard scanner 43 interrupts acentral processing unit (“CPU”) 47 via an interrupt line 49. CPU 47responds to the interrupt by placing the address of keyboard scanner 43upon an address bus 51. In response, an address decoder 53 generates aselect signal on a scanner select line 55 which enables keyboard scanner43. CPU 47 then places a read enable signal on a read line 57. Keyboardscanner 43 responds by placing the digital key-stroke code on a data bus59. Next, CPU 47 reads the key-stroke code into an internal key-strokeregister 61 and increments a print count register 63, a transmit countregister 65 and a memory pointer register 67. The count in register's 63and 65 represents the back-log of key-stroke codes to be printed andtransmitted, respectively. Immediately after the key-stroke code hasbeen read, keyboard scanner 43 removes the interrupt and awaits thecourt reporter's next stroke.

Having retrieved the key-stroke code, CPU 47 must first save the codeand then attempt to transmit and print it. Specifically, from thecontents of memory pointer register 67, CPU 47 computes the nextavailable address in RAM 37 and places that address onto address bus 51,while placing the contents of key-stroke register 61 onto data bus 59.In response to the address on address bus 51, address decoder 53produces a select signal on RAM select line 69, selecting RAM 37.Afterwards, CPU 47 places a write enable signal upon a write line 71,and RAM 37 responds by storing the key-stroke code.

Having saved the key-stroke code, CPU 47 enters a prioritized wait-mode,awaiting an interrupt signal from: 1) keyboard scanner 43 alonginterrupt line 49, signifying that another key-stroke code has beendetected and generated; 2) communication port 39 along interrupt line73, signifying that port 39 is ready to receive a key-stroke code fortransmitting; or 3) print actuator 33 along interrupt line 75,signifying that actuator 33 is ready to receive another key-stroke forprinting. Interrupt servicing priority is given to scanner 43, followedby port 39 and actuator 33.

This servicing scheme does not provide first in first out servicing ofinterrupts. Instead, at the conclusion of servicing any interrupt, CPU47 enters the prioritized wait-mode and, first, unmasks interrupt line49 from scanner 43. If an interrupt is present, CPU 47 exits thewait-mode and services scanner 43. Otherwise, with no interrupt fromscanner 43, CPU 47 examines the backlog count in registers 63 and 65. Ifneither contain a backlog, CPU 47 waits solely for an interrupt fromscanner 43.

If one or both contain a backlog, priority is given to the register withthe largest backlog or to transmit count register 65 if the registersare equal. With this priority, CPU 47 unmasks the correspondinginterrupt line. If the corresponding device has provided an interruptsignal signifying that it is ready, CPU 47 will exit the wait-mode andservice that device. Otherwise, if that device is not ready, CPU 47 willunmask the final interrupt line. At this point, CPU 47 will service thefirst interrupt that occurs. Finally, before servicing any interrupt,CPU 47 masks all interrupts.

Having described the prioritization scheme and the servicing of keyboardscanner 43, we move on to servicing communication port 39. As previouslydetailed, during the servicing of scanner 43, transmit count register 65contains a backlog count of at least one, signifying the need totransmit. The count in register 65 directly represents the binary numberof key-strokes that have been stored but not transmitted. If register 65contains a binary value of one at the time CPU 47 responds withservicing, CPU 47 will place the address of communications port 39 ontoaddress bus 51, while placing the contents of key-stroke register 61(the recently received key-stroke code) onto data bus 59. Addressdecoder 53 responds by placing an enabling signal on port select line77, selecting communications port 39. CPU 47 next places a write enablesignal upon write line 71, causing the key-stroke code on data bus 59 tobe written to communication port 39.

Upon receiving the key-stroke code, port 39 removes the interrupt signalfrom interrupt line 48 until the key-stroke code is transmitted to CATsystem 13 via communication link 19 (see FIG. 1). CPU 47 exits theservicing routine and enters the prioritized wait-mode.

If, however, transmit count register 65 reveals a greater backlog thanjust one, CPU 47 must retrieve the oldest un-sent key-stroke code fromRAM 37 before sending it to communication port 39 as described above. Toretrieve the oldest un-sent keystroke code, CPU 47 calculates theaddress of the oldest keystroke code by subtracting the contents oftransmit count register 65 from memory pointer register 67 and placingthat value on address bus 51. Address decoder 53 responds by selectingRAM 37 via RAM select line 69. Next, CPU 47 provides a read enablesignal on read line 57 which causes RAM 37 to place the oldest un-sentkey-stroke code onto data bus 59. CPU 47 then reads this key-stroke codeinto an auxiliary register 79, completing the retrieval. Finally, thesame steps described above are engaged in by CPU 47 to write thiskey-stroke code (stored in auxiliary register 79) to communication port39.

In an identical manner, responding to interrupt line 75, CPU 47 servicesprint actuator 43 utilizing address decoder 53's selection of actuator33 (via actuator select line 81) and either print count register 63 orauxiliary register 79 as necessary. The details therefore need not berepeated.

A read only memory (“ROM”) 83 provides the instructions or program forCPU 47 which defines the operation described above. This program alsodefines the initialization and mode of operation of stenographicrecorder 11. ROM 83 is accessed much the same way RAM 37 is accessed,using ROM select line 85 from address decoder 81.

Additionally, although not shown, RAM 37 can be backed-up withbatteries, providing a relatively permanent storage of the record forlater retrieval. Otherwise, stenographic recorder 11 can service a diskdrive system (not shown) which could either operate in parallel with RAM37 but at a lower priority in the servicing scheme, or completelyreplace RAM 37. With either scenario, the disk drive system would createthe permanent storage on a diskette.

As a final matter, while in the prioritizing wait-mode, CPU 47 canevaluate the backlog of the devices which are being serviced and readthe status of a severely back-logged device to identify a problem.Warning messages can then be sounded or displayed to the court reporter.This might also be accomplished by using additional interrupt linessignaling malfunctions.

FIG. 3 is a detailed view of the interconnection of stenographicrecorder 11 and CAT system 13. As previous described, in response to thecourt reporter stroking keyboard 31, recorder 11 creates a key-strokecode and transmits this to the CAT system 13 (the reporter's terminal)along communication link 19. Key-stroke codes received are displayed intwo different formats on a screen 100 of CAT system 13. In a tape window101 which is shaped and sized much like paper tape 35, alphabeticrepresentations for received key-stroke codes are displayed in a line byline fashion wherein each key-stroke code is represented on a singleline. Tape window 101 thus provides an electronic version of a papertape which is generally in the format familiar to a court reporter.

Screen 100 also includes a transcription window 103 which displays thealphabetic and numeric text representative of the spoken words, i.e.,displays the transcribed version of the key-stroke codes received fromrecorder 11. Through user selection, the windows 101 and 103 can beconfigured to update the screen 100 upon complete transcription ofeach: 1) word; 2) sentence; 3) Q & A; or 4) Q or A. The display can alsobe chosen to operate in either scrolling or paging fashion. For example,with a preferred display selection, the transcription window 103 willonly display the current Q & A as it is developed on a sentence bysentence basis. After completing the transcription of the first sentenceof a new question, the CAT system 13 clears the currently displayed Q &A (the old Q & A) from the window 103, and displays the transcribedalphabetic and numeric text of the first sentence of the new question.As each additional sentence of the new Q & A is received andtranscribed, it too is added to the window 103. After fully transcribingthe new Q & A, window 103 holds the display of that new Q & A until theend of the first sentence of the next Q & A; upon which, this cyclerepeats.

To set up the mode of updating the windows or to access any of a varietyof other CAT system 13 commands, the screen 100 also includes a commandline 105. In one embodiment, the command line 105 lists several one-wordcommands displayed horizontally in a single line of text. The commandslisted by the command line 105 can be selected directly using a mouse,pen, fingertip or other input device (none of which are shown) orindirectly by simultaneous pressing the “alt” key and the first letterof the desired command on the keyboard 107. Any other mode or method ofcommand selection involving, for example, button bars, pull-downmenuing, or icons might also be used.

As previously stated, a court reporter strokes the stenographic recorder11 in response to the spoken word. Generally, a given word may demandeither a single stroke or multiple strokes based on the complexity ofits phonetic composition. Upon completing the strokes necessary for agiven word, the court reporter immediately begins stroking the nextword. No space bar or other word separator is stroked between words. Asa result, all of the strokes run together. At the end of a sentence, thecourt reporter does stroke recorder 11, however. The CAT system 13operates on this sequence of key-strokes to determine the beginning andend of each word.

Providing the foundation of the transcription process, the courtreporter uses a personalized cross-reference library which lists thereadable text of every word that the court reporter has translated todate. Each of these words are associated with the correspondingstenographic key-stroke or sequence of key-strokes that represents thatword. This cross-reference library can be displayed in a visual manner,but is stored in binary tree structure for efficient searching.

Upon receiving the first key-stroke code of the sentence, CAT system 13begins transcription. This first code is compared to all of the possiblefirst stroke codes within the court reporter's cross-reference library.If a complete cross-referenced word is found, CAT system 13preliminarily treats this as the entire word. Then, upon receiving thesecond key-stroke code in the sentence, CAT system 13 treats this secondkey-stroke code as the beginning of another word and looks again to allof the possible first stroke codes within the library. If that code isnot found, CAT system 13 must back-track. The first and second codes arecombined and are then compared to all of the possible first and secondstroke codes within the library. If a complete cross-referenced word isfound, CAT system 13 replaces the shorter, single stroke word with thisnew, two stroke word, and preliminarily treats this as the entire firstword. Next, a third code is received and is similarly treated as thebeginning of another word, and the cycle repeats.

By the end of the sentence, CAT system 13 provides the besttranscription possible. Key-strokes that still remain untranscribed atthat point are translated using a phoneme library into readable phonemerepresentations of those key-strokes to aid both the court reporter and,more importantly, those lay-persons down-line. By pronouncing thesephonemes, the lay-person might identify the actual word or words whichthe cross-reference library did not contain. Substituting the phonemerepresentation with the actual word or words causes the cross-referencelibrary to be updated.

Once fully transcribed using the cross-reference and phoneme libraries,the entire sentence is communicated down-line to attorney terminals 15and 16 via communication links 21 and 20, respectively.

More specifically, FIG. 4 a is a diagram representing the association ofdata fields into a record which is used as a basis for the overallbinary tree, data structure used in transcription according to thepresent invention. In particular, CAT system 13 utilizes a linked-listarrangement of two types of data records: a key-stroke code listing(KCL) record 151 and a corresponding text (CT) record 153. These twotypes of records provide the storage structure for the court reporter'spersonalized cross-referencing library.

Basically, CAT system 13 uses records 151 and 153 to associate eachindividual key-stroke code with as many subsequent key-stroke codes torepresent a given transcribed word. KCL record 151 associates: 1) alisted key-stroke code (LKC) field 155, for storing a specifickey-stroke code; 2) a listing optimizing counter (LOC) field 157, forstoring a count used for optimizing future searches; 3) a firstKCL.sub.—record pointer field 159, for storing a pointer to the next KCLrecord on this level; 4) a CT record pointer field 161, for storing apointer to an associated CT record; and 5) a second KCL.sub.—recordpointer field 163, for storing a pointer to a correspondingKCL.sub.—record at the next listing level down. Similarly, CT record 153associates: 1) a CT string field 165, for storing a string of text; 2) atext optimizing counter (TOC) field 167, for storing an additional countused for optimizing transcription; and 3) a CT record homonym pointerfield 169, for storing a pointer to another CT record containing ahomonym to the contents of CT string field 165.

FIG. 4 b is a detailed diagram representing a portion of the overallbinary, linked-list data structure used by CAT system 13 intranscription. KCL records 200, 201 and all KCL records (not shown)directly to the right and left of records 200, 201 constitute a firstlisting level. This first listing level is no more than a linked-list ofthe each beginning stroke in the stenographic representation of thewords held in the court reporter's library. Upon receiving the firstcode of a sentence, that code is compared with each code stored in eachKCL record on the first listing level. If, for example, the receivedcode does not match the stored code in KCL record 200, KCL record 200points to the next KCL record 201 for a similar comparison to the codestored therein. In this manner, by stepping through the first listinglevel, a matching KCL record can be found.

Assuming that the code stored in KCL record 200 does match the firstkey-stroke code received, the associated CT records 205 and 207 can beaccessed to retrieve readable cross-referenced text. KCL record 200 canalso be used as a back-tracking point. If the word transcribed requirestwo key-strokes, KCL record 200 points to the second listing level. Thissecond listing level is specifically associated with KCL record 200 andbegins with KCL records 202 and 203 followed by all KCL type records(not shown) to the right of record 203. Any second code received whichfollows a first code which matches that stored in KCL record 200 iscompared to the codes stored in the KCL records on the second listinglevel. KCL record 204 represents yet a third listing level under thekey-stroke sequence stored in record 200 and 203, and so on. CT recordsmay or may not be associated with a given KCL record. KCL record 202exemplifies such a situation. Additionally, only a single CT record isgenerally associated with a single KCL record, such as is shown with KCLrecord 203 and CT record 209. Only when homonyms exist will there bemultiple CT record association as shown with KCL record 200 and CTrecords 205 and 207.

Particularly, upon receiving a first key-stroke code from stenographicrecorder 11, CAT system 13 begins a transcription expedition by parsingthrough a first listing level in an attempt to find a matching KCLrecord. If a matching KCL record is found which has an associated CTrecord transcription, CAT system 13 decides there is a preliminarytranscription and treats the second (next) code received as thebeginning of a new word, parsing the first listing level for a matchingKCL record.

If a matching KCL record is found for first code received yet without anassociated CT record transcription, the second code received is comparedfor a matching KCL record at the second listing level, if a second levelbeneath the matching KCL record on the first level exists. If a match isfound at the second listing level with an associated CT recordtranscription, the third key-stroke code is treated as the beginning ofa new word, and parsing at the first listing level repeats.

If after a match with associated transcription text a key-stroke yieldsa dead end, back-tracking must occur. The requires picking up where thelast successful parsing expedition left off, searching deeper listinglevels to see if what had been considered an entire word is really onlya portion thereof. If a match is found with an associated CT recordtranscription, that transcription at that deeper listing level isstored, and the next key-stroke code received is treated as thebeginning of a word and the cycle repeats.

With each successive, unsuccessful parsing round, the previouslydescribed transcription process becomes more and more complex with manyparallel and nested transcription pathways being considered. Ifavailable, the first completely transcribed sentence found iscommunicated to attorney terminals 15 and 16.

If a completely transcribed sentence is unavailable, a second phase oftranscription, the phoneme transcription phase, is invoked. In thisphase, as illustrated by FIG. 5 a, each untranscribed key-stroke iscross referenced into a phoneme replacement table or phoneme library 220to obtain readable phoneme text. Specifically, the CAT system 13 parsesthrough a series of listed key-stroke codes held in alisted.sub.—keystroke.sub.—code field 223 of the phoneme library 220 tolocate the key-stroke code entry which corresponds to the untranscribedkey-stroke code. Once located, the CAT system 13 extracts the readablephoneme text stored in a corresponding.sub.—phoneme field 221 of thephoneme library 220. Thereafter, extracted text is substituted for theuntranscribed key-stroke in the sentence. Once all of the untranscribedkey-strokes of the sentence have been replaced with correspondingphoneme text, the sentence is communicated down-line to the attorneyterminals. For example, if the CAT system 13 receives a key-stroke coderepresenting the key-stroke “W A Z” for which no counterpart exists inthe cross-reference library, the CAT system 13 parses thelisted.sub.—key-stroke.sub.—code fields 223 and locates the specificentry in a field 225. The corresponding readable phoneme text entry“was” in a field 227 is then extracted and substituted for theuntranscribed key-stroke code in the sentence.

In addition, if homonyms are found, signifying multiple CT recordassociation, they are displayed within parenthesis. For example, thehomonyms “their”, “there” and “they're” ( ) in the question “Were youthere?” would be displayed as:

Q. Were you (there.vertline.their.vertline.they're)?

Although readable, to correct such common homonym transcriptionproblems, another level of transcription is contemplated using so called“artificial intelligence” techniques for determining the correct wordgrammatically in the context of the given sentence.

As a final matter associated with the data structure, optimizingcounters are placed in each record so as to provide optimization of theoverall structure. Each time a record is used in the final translatedsentence, the associated optimizing counters are incremented. At the endof a deposition, the court reporter can run an optimizing routine whichsorts records in each linked list based on the count in the optimizingcounters. In this way, the most common matching KCL records areencountered first, speeding up the parsing response time. Similarly, themost common homonyms will appear first in the parenthesis.

Referring back to FIG. 1, CAT system 13 communicates a sentencedown-line to terminals 15,16,17,18 when that sentence has been fullytranscribed. Full transcription takes place whenever CAT system 13receives an end of sentence key-stroke code (“EOS”) from stenographicrecorder 11. If, however, while in mid-sentence, the witness pauses tothink, an EOS will not be received. To satisfy an attorney who wants totake this opportunity to review what has been stated thus far, anartificial EOS can be generated. An artificial EOS is generated ifeither: 1) the pause between the court reporter's key-strokes exceeds apreset time limit (programmable from zero to one hundred and twentyseconds); or 2) an attorney down-line presses the space bar on terminal15, 16, 17 or 18 indicating via the corresponding communication links20,21,22,23 the desire to display what has been received thus far. Inresponse to an artificial EOS, CAT system 13 completes the translationas if an EOS has been received, attaches an artificial EOS to the end,and sends the transcribed “partial” sentence down-line. Thereafter, ifthe EOS is the very next key-stroke received, CAT system 13 merelycommunicates an EOS down-line to replace the artificial EOS.Alternatively, if further non-EOS key-strokes are received beforereceiving an EOS, CAT system 13 completes the transcription of theentire sentence with the EOS and sends it down-line. Terminals15,16,17,18 over-write the “partial” sentence with the complete sentencereceived. In the alternative, terminals 15,16,17,18 might instead merelydisplay partial sentences, only writing full sentences to file.

Depositions involving participants in two or more languages are alsocontemplated by the present invention. In a first embodiment, thetranscription process operates as previously described by providing aword for word transcription of the spoken question in a first language.Each of these words are then cross-referenced into a second languagetable to generate a sentence in a second language. Although the use ofthe second language table can be assigned to each attorney terminaldown-line, in a preferred mode, the CAT system 13 performs the task.Once translated, the CAT system 13 transmits the second languagesentence down-line to viewing participants who understand that secondlanguage. By using additional language tables, the CAT system 13 canprovide translations for as many languages as is needed.

If the CAT system 13 performs the translation task, the CAT system 13also manages the specific down-line routing of first and second languagesentences, automatically. To accommodate automatic routing, allparticipants are required to identify their viewing language and theirrole in the deposition upon logging in to the system 13. The CAT system13 stores the identified information in a log-in list. By referencingthe log-in list, the CAT system 13 automatically configures itself so asto provide each participant with a translation in their appropriatelanguage.

An exemplary second or foreign language table 231 is illustrated in FIG.5 b. Using the language table 231, the CAT system 13 parses the textentries of a first language in a series ofidentified.sub.—CT.sub.—String fields 233 to locate the transcribed textof a word in a first language. Once located, the text of a secondlanguage is extracted from a corresponding.sub.—translation field 231and placed into a second language sentence. Each transcribed word in thefirst language undergoes this process so that a second language sentencecan be fully constructed. Thereafter, both the first and second languagesentences are transmitted down-line to the appropriate participant. Forexample, the word “good” located in a field 235 in a first language,English, would be directly substituted by the word “bueno” of a secondlanguage, Spanish, contained in field 237. Although translations wouldnot always be exact because of word context, the use of a humantranslator could be minimized. Moreover, as a supplement to the languagetable 230, word context analysis might also be implemented to increasethe accuracy of the translations.

If the down-line participants that speak during the deposition all sharea common language, the court reporter need only understand the commonlanguage and provide a cross-reference library and a phoneme library forthat language. Alternate language translations for non-speaking,down-line participants, are provided using as many different foreignlanguage tables as needed, as described above. Routing and languageinformation is retrieved from the log-in list.

If, however, the down-line participants that speak during the depositionshare two or more languages, in a preferred embodiment, a multi-lingualcourt reporter provides a cross-reference library, and phoneme libraryfor each of the languages being spoken. Additionally, for each spokenlanguage, language tables providing translation into each of themultiple languages are also necessary. During the translation process,when the multi-lingual court reporter indicates the beginning of aquestion or answer, or otherwise identifies the speaker, the CAT system13 automatically selects the appropriate cross-reference and phonemelibraries to transcribe the spoken language, then accesses theappropriate foreign language tables to construct sentence translations.The resulting translations are automatically routed down-line. The CATsystem 13 performs the automatic selection of libraries and tables androuting with reference to the entries in the log-in list.

Although the use of multiple phoneme libraries is preferred in amultiple language deposition, they may be completely ignored. Otherwise,when attempting to translate phoneme text into a second language, thephoneme text should be replaced by equivalent phoneme text in the secondlanguage. Therefore, pronunciation of the spoken word would sound thesame in either language. Because with phoneme text the actualtranscription of a word is unknown, a second language translation cannotbe found. Therefore, as a substitute, a pronunciation for each of theparticipants in the phonemes of their language which sounds like thespoken word in the first language is provided.

Referring to FIG. 6 a, examining attorney terminal 15 includes a screen251 and a keyboard 253. Screen 251 is split into a transcription window255 and a communication window 257, having a common command line 259.The display of window 255 is very similar to that of window 103 in CATSystem 13 (FIG. 3), displaying the transcribed text and having the samesetup response choices. This transcribed text is received by terminal 15from CAT System 13 in coded form along communication link 21 in asentence by sentence fashion as described above.

Even though transcription is received on a sentence by sentence unitbasis, in the preferred mode of response, the sentences units receivedare not displayed in window 255 until an entire question or respondingentire answer has been received. Three indicators are used to signifythe end of a question (EOQ) or end of an answer (EOA): 1) a courtreporter's key-stroke signifying the beginning of an answer, clearlyindicating an EOQ and visa versa; 2) the expiration of a preset delay(programmable from zero to one hundred and twenty seconds) since thelast received sentence, providing an artificial EOQ or EOA; and 3) theexamining attorney pressing a space bar 261, indicating a desire to seewhat has been received thus far and providing an artificial EOQ or EOA.Because the second and third indicators only signify that a question oran answer has possibly ended, they are updated upon receiving a true EOQor EOA.

As illustrated in FIG. 6 b, the question begins in window 255 with thesymbol “Q” followed by the question. For example, FIG. 6 b shows thedisplay:

Q. When did that occur?

Afterwards, upon receiving the entire transcribed answer (signified byan EOA or an artificial EOA), the answer appears in window 255 directlybelow the question. The answer appears on window 255 following thesymbol “A”. For example, FIG. 6 b shows the answer:

A. Around noon.

If the remaining room in window 255 is such that the Q & A cannot besimultaneously displayed, the answer is displayed at the bottom-mostportion of window 255 while the remaining room is allocated to the endof the question.

Each Q & A remains displayed until the next transcribed question isreceived in its entirety. This is defined by either the receipt of anEOQ or an artificial EOQ. Thereupon, the last Q & A is cleared fromwindow 255 and the entire next question is presented at one time, andthe Q & A cycle repeats.

By permitting only the display of one Q & A unit at a time,transcription window 255 remains uncluttered, and the examining attorneycan easily focus on the pending question and/or the last answer. Thus,for each successive Q & A unit, window 255 is cleared and then displaysthe full, transcribed text of the question followed by its correspondinganswer.

In a preferred mode, each Q & A unit is associated with a differentnumber so as to provide a reference to specific Q & A's. The numbers aregiven in consecutive order for ease of reference. As shown in FIG. 6 b,the Q & A displayed is associated with the number 57, signifying thatthe displayed Q & A is the 57th Q & A received. In its entirety, window255 shows:

*57 Q. When did that occur?

57 A. Around noon.

Keyboard 253 includes an up arrow key 263 and a down arrow key 265 forscrolling through the Q & A's. For example, stroking the up arrow key263 causes scrolling upwardly in a line-by-line fashion bringing theprevious Q & A into display on window 255. For faster movement between Q& A's, keyboard 253 provides a page up key 267 and a page down key 269.In response to a single stroke, keys 267 or 269 cause paging betweenentire Q & A's. In addition, keyboard 253 includes a “home” key 270 andan “end” key 271. No matter which previous Q & A is currently beingdisplayed, actuation of end key 271 causes window 255 to display eitherthe last full Q & A or, if no answer has been completed, to the lastpending Q. Similarly, home key 270 causes window 255 to display thefirst Q & A of the proceeding.

Keyboard 253 also includes a “mark” key 273 which permits any Q & A tobe marked for later reference. The marked Q & A is indicated in thewindow by an asterisk symbol appearing on the screen to the left of theQ symbol as shown in FIG. 6 b. Associated therewith, keyboard 253 alsoincludes a “quick-up” key 275 and a “quick-down” key 277 which permitscrolling only through the asterisked marked questions.

In particular, if an important answer or definition has been given bythe witness, the examining attorney can mark the Q & A with an asterisk.Whenever the attorney wishes to refer back up to that answer, he or shemay merely press quick-up key 275 to recall to the display window 255the marked questioned with its answer. The attorney may also desire torefer back up to a Q & A marked several marked Q & A's ago. Toaccomplish this, successive strokes to quick-up key 275 pages backwardthrough previously marked Q & A's. Although stroking end key 271 returnsthe display in window 255 to the pending question, the examiningattorney may desire to sequentially page down through the marked Q & A'sas they occurred. The quick-down key 277 provides this ability. Inaddition, marked Q & A's can be referred to after the deposition inpreparation for the next day's deposition. As understood, pertinentquestions may also be marked at a time outside of the deposition, forexample, later that evening at the hotel room before the next day'sdeposition.

Accessing a specific Q & A can be achieved through the associated numberassigned to that Q & A. By pressing a “number” key 278 followed by thenumber previously associated with a specific Q & A, terminal 15 locatesand displays that Q & A in window 255. A specific Q & A can also belocated through a lexical scanning capability accessible through ascan-up key 279 and a scan-down key 281. For example, by pressing scanup key 279 followed by a lexical searching template, attorney terminal15 scans backward through the previous Q & A's in an attempt to findtext matching the lexical template. If a match is found, that Q & A isdisplayed on window 255, else a “no match found” message is received.Afterwards, stroking scan-up key 279 again pulls up the most recentlyused lexical template. By merely stroking scan-up key 279 withoutmodification to the template continues the search further upward throughprevious Q & A's. Scan down key 281 works in a similar manner butscanning in the opposite direction.

In general, the lexical template format and features are similar tothose provided in legal searching databases familiar to most attorneys.These templates can be constructed using logical operators, wild cardcharacters, quotes, parenthesis and brackets with specific text to belocated. The lexical template ordinarily must be satisfied by a single Q& A. To over-ride this “default” condition, a delimiter must be used.For example, placing a “/A” or a “/Q” delimiter within the template,directs attorney terminal 15 to satisfy that template, or portionthereof, entirely within a single answer or a single question,respectively. A “/QA2” delimiter expands the search to within twoconsecutive Q & A's; while a “/QA3” corresponds to within threeconsecutive Q & A's, and so on. Similarly, a delimiter of “/A3” or adelimiter of “/Q5” respectively require satisfaction of the lexicaltemplate to within three consecutive answers or within five consecutivequestions. Delimiters “/W4”, “W5Q”, or “W3A” require the respectivesatisfaction within: 1) four consecutive words in either a single Q orA; 2) five consecutive words in a question only; or 3) three consecutivewords in an answer only.

In addition to searching only the current transcript, lexical searchingcan also be applied to all other evidence in the lawsuit. The terminal15 permits either locally or remotely stored transcripts recorded atprevious hearings or depositions to be selected and searched through thecommand line 259. Similarly, all other case evidence such as thepleadings documents or documents produced through discovery may bescanned-in, converted to text using character recognition software, andmade available for the lexical searching. Once located through a search,the scanned-in image can be retrieved and displayed or printed for useas an exhibit at the ongoing deposition.

Remotely located legal databases such as Lexis® or Westlaw® can also beaccessed and searched through the command line 259 during thedeposition. Searching other sources may be used to identifydiscrepancies in the witness's current testimony, or to help clarify thelaw in an unexpected area so that appropriate questioning can bedirected.

Although boolean logic is used for formulating lexical search requests,a natural language front end is also contemplated. The attorneyterminals could provide for user preference, invoking the boolean logicor natural language front end upon request. The lexical formulation fora search is identical whether searching any of the case evidence orlegal databases, thus providing uniformity of searching.

The specific placement of the various keys shown on keyboard 253 is notcritical. In fact, the illustrative placement in FIG. 6 b is merely setforth for ease of explanation. Additionally, the functionality of thesekeys may be accessed directly from the command line 259 as explainedabove.

Pressing a tab key 285, or through direct selection with a mouse, pen orfinger-tip, causes toggling between transcription window 255 andcommunication window 257. The communication window 257 provides a visualdisplay in either a stack mode and an edit mode. As illustrated, in thestack mode, communication window 257 displays the first line of eachcommunication received. These communications originate from threesources: 1) the examining attorney's keyboard 253—personal notes “From:Self” or messages “To:” others; 2) the examining attorney terminal15—through artificial intelligence (“AI”) algorithms “From: AI” whichconvey potential objections or through scheduler messages; and 3) theother terminals on the network such as the associate examining attorneyterminal 17—messages “From:” others. While in the stack mode, window 257can display all of the communications from one source, all sources, orany combination of the three sources.

To fully read any communication, the examining attorney merely selectsthe desired communication from the stack of messages illustrated in thewindow 257, causing the window 257 to display of the full text of thecommunication in an edit mode. Communications can also be created,modified, deleted, copied, printed, or communicated to the otherterminals, such as associate examining attorney terminal 17 via thecommunication window 257.

In this embodiment, one side's message exchange along that side'scommunication Link cannot be intercepted by the opposing attorney's notonly because the communication links are isolated, but also because ofthe controls placed on the CAT system 13. The CAT system 13 can onlyconsider relaying communications from one side to the other if aspecific request therefor is received. Moreover, even if a specificrequest is received, the CAT system 13 cannot relay communicationsunless specific permission has been given to receive suchcommunications. In particular, an attorney on one side sends acommunication to the CAT system 13 with a specific request that thecommunication be relayed a recipient on the opposing side. Oncereceived, the CAT system 13 complies to the request only if therecipient on the opposing side grants permission for the relaying. Togrant or retract permission, each side sends a message to the CAT system13, at log-in or at any subsequent time, indicating whether or notrequests to relay messages from the other side should be honored. Ifnot, the CAT system 13 responds to each request by sending ado-not-disturb (DND) message to the requesting terminal. If so, the CATsystem 13 responds by relaying the message from the sending side'scommunication link to the receiving side's link.

In either the edit or stack display modes, communication window 257interfaces with a mouse, keyboard 253 and associated keys in much thesame way as does transcription window 255. For example, up and downarrow keys 263 and 265 and page up and down keys 267 and 269respectively permit scrolling and paging through the stack or text.However, the scan-up and scan-down keys 279 and 281 operate in aslightly different way. In particular, as a default, lexical scanningonly parses the currently displayed stack, i.e., the currently selectedcommunication sources. A global command delimiter “/G” overrides thedefault so that all communications from all sources can be scanned. Thefollowing use of delimiters can override the default: a “/M” formessages only; “/N” for notes only; “/AI” for AI objections only; or forcombinations of these fields—for example “/MN” for messages and notesonly.

The examining attorney may also choose to modify a significanceindicator, preferably a fuzzy-logic type slide-bar icon, which causesthe attorney terminals to associate a level of priority with eachmessage, note, or other communication received. The significanceindicator may default at an average oar middle setting and need not beadjusted. If selected as a set-up option, the significance indicator canbe used, along with the age of the communication, to adjust the order ofthe stack of communications. The significance indicator can also be usedto associate a window or window border color with the level of prioritythe communication carries. When set to a maximum value the significanceindicator can also be used to send a flashing message to anotherterminal, indicating immediate attention is desired. This functionalityexists on each of the terminals in the transcription network.

The examining attorney may also type to himself his own notes as thedeposition is progressing. These notes may serve useful in that they maybe reviewed later that evening in order to prepare for the next day ofdeposition, or they may be used at a later time during the depositionwhen a question becomes ripe. Most attorneys take notes during adeposition. Those who type proficiently may take notes with keyboard253, and those who do not might use pen-based input as described morefully below in relation to FIG. 13.

Alternatively, the examining attorney may type or pen notes into histerminal before the deposition begins. In particular, the attorneyterminals 15 and 17 provide an outline mode for hierarchically outliningby subject category and sub-category a series of ordered questions toask the witness or to act as a reminder as to certain areas ofquestioning that must be covered during the examination.

Referring to FIG. 7 a, while off-line, upon entering the outline mode,the examining attorney uses the attorney terminal 15 to create aconventional outline of the names of categories and subcategories to becovered at a deposition using an outline window 285. As illustrated, anarea of law such as “patent law” is entered as a hierarchical entrypoint by assigning it a Roman numeral as is in typical outlines. Eachsubcategory of patent law such as infringement, invalidity, laches,etc., is then entered thereunder. Similarly, the attorney continues toenter sub-subcategories and so on until all of the categories of patentlaw that need to be discovered are in the outline. Next, the attorneyenters questions or notes under the appropriate categories in theoutline. For example, under “best mode” the attorney may enter thequestion: “Did you disclose the most recent version of your invention inthe patent application?” After all of the questions (or notes) have beenentered, the attorney then estimates the amount of time that should bespent on each category. This time is entered through a command line 283for use by a scheduler (described below in more detail).

After entering the entire outline, questions and estimated times, theattorney invokes a toggle command on the command line 283 which causesthe outline structure to enter the graphical tree-structured displaymode illustrated in FIG. 7 b. In this mode, the outline window 285displays the created outline in a tree-structured format. Each categoryand subcategory can be easily discerned as well as the total estimatedtime to finish each hierarchical level. For example, in a block 287, thearea of law, patent law, can be found to have an estimated time of eight(8:00) hours to complete. This time is comprised of one hour and thirtyminutes (1:30) to complete “laches” at a block 289, five hours (5:00) tocomplete “invalidity” at a block 291, and one hour and thirty minutes(1:30) to complete “willfulness” at a block 293. In the preferredembodiment, the underlying questions or notes associated with eachcategory or sub-category are not displayed in the tree-structureddisplay. Toggling between the standard outline display and thetree-structured display can be repeated until preparation for thedeposition is complete.

In addition to generating entire outlines from scratch, standard,reusable outlines pertaining to specific areas of law may be effectivelyutilized as a default. Once the outline is created, it may be re-used insubsequent depositions by merely adding or deleting questions whereapplicable.

Pretyped questions are used during the deposition by both the examiningand defending attorneys to direct the proceedings. In addition, notescan be added detailing the current state of the law in each subcategoryfor reference during a deposition.

Referring back to FIG. 6 a, while in the outline mode during adeposition, the configuration of windows on screen 251 of the examiningattorney's terminal 15 is identical to that described above with oneexception. Instead of displaying Q & A's, the window 255 is used for anoutline mode display. Particularly, whenever an outline command isselected from the command line 259, the window 255 displays thetree-structured outline illustrated in FIG. 7 b. From this outline, theattorney may choose to: 1) toggle to the standard outline, illustratedin FIG. 7 a, to update the outline; 2) evaluate and, if necessary,change the time estimates based on the deposition progression; or 3)select a category of questions and notes for immediate use.

By double-clicking the mouse button on the desired category (or throughcommand selection via the command line 259), the pretyped questions andnotes relating to the selected category are assembled onto a stack anddisplayed in the communication window 257 for immediate use. Upon usersetup selection, the pretyped communications can be pushed onto thestack or be placed into a new independent stack which is solelydisplayed on the window 257. With the later selection, an individualstack is constructed for each and every subcategory in the outline.

As with off-line preparation, during the deposition, notes may bemodified or added. Similarly, new categories may be added so thatexisting notes may be placed thereunder. To move an existing note, thenote need only be selected via the window 257 and transferred byselecting a category via the window 256.

In an alternate embodiment, color coding is used to aid theidentification of areas of remaining inquiry. For example: a greycategory name indicates that all of the notes thereunder have beenaddressed; a yellow category name indicates that all of the notesthereunder need to be addressed; and a blue category name indicates thatsome of the notes thereunder need to be addressed. Similar color codingcan be used for the specific notes. Colors can change automatically,upon detecting that all notes have been used, or manually by theattorney upon exhausting an inquiry without using all the questions ornotes.

In addition, the terminals 15 and 17 record the range of the Q & Anumbers which occur while within each specific category or subcategory.The range of Q & A's recorded can thereafter be used to aid in thereview of areas of inquiry which are scattered throughout a transcript.

Objections to form must be seasonable, therefore timing is of utmostimportance. To speed up the objection process, AI software routinesanalyze the form of each question and the content of the answer duringand immediately after translation to provide various potentialobjections that the examining attorney may want to take intoconsideration in attempting to achieving good evidentiary form. Forexample, a search is made on each question for phrases such as “yousaid” or “you stated” or “you say”. If found, the AI routinesimmediately send a potential objection to the communication windowsindicating that the question possibly mischaracterizes the witnessesearlier testimony. In response to the communication, the defendingattorney may object, and may also initiate a search for themischaracterized statement by opening the communication and pressing thescan up key 279. In this way, mischaracterizations may be avoided orcorrected. Similarly, for example, answers are analyzed with focus onwords such as “said” and “stated” to identify potential hearsay therein.The examining attorney may choose to object or ignore the communication.

As illustrated in FIG. 8, associate examining attorney terminal 17includes a keyboard 301 having alphabetical keys arranged in a standardtyping keyboard key pattern. As with the examining attorney's terminal15, the associate examining attorney terminal 17 includes a screen 303which is split into a transcription window 305 and a communicationwindow 307 which have a common command line 309. While in the depositionmode, the terminal 17 displays the alphabetic and numeric textrepresenting the spoken words in the transcription window 305, and thewindow 307 provides for input and display of messages, notes, etc. Inthe outline mode, the window 305 performs the hierarchical outliningfunctions described more fully above in relation to the terminal 15.

The associate attorney may also send messages to the examining attorneyas the deposition is being carried out. The functionality of suchcommunication is identical to that described previously regardingcommunication in the opposite direction with one enhancement. Theassociate's terminal 17 provides the capability of viewing, modifying,and sorting the communication currently present on the attorney terminal15. Such capability allows the associate attorney to help organize theterminal 15, which has added importance for an examining attorney withlittle computer literacy.

AI objections are always associated with specific questions or answers.Messages may be associated upon selection via the command lines so thatthe recipient can place the question in context. Notes can be similarlyassociated. Important messages, notes, or objections can be recorded ina supplemental file by using the annotate command from the command line.Annotating creates a file of annotations which may or may not beassociated with a specific Q & A, which may be stored with thetranscript for later retrieval and review.

A demeanor marker command also provides for annotating a specific Q & A.Noticing that the witness turns red or begins to stutter, fidget, orraise their voice in response to a specific question, an attorney canselect the demeanor command which presents a pretyped list of demeanorindications such as “fidgeting” or those listed which may be immediatelyassociated with the current Q & A and stored as an annotation to thetranscript. Similarly, a conferral command can be selected to annotatethat the witness and opposing attorney conferred off the recordregarding an associated Q & A. Additionally, although not generallyuseful during the deposition because they may be overheard, voicerecorded notes or annotations may also be added by an attorney reviewingthe transcript away from the deposition. Such annotations may beselectively displayed or played back via the attorney terminal's commandline.

In the communication windows of each terminal, a stack containing aseries of common pretyped messages (“common message stack”) can bequickly accessed for transmittal. For example, in the communicationwindow 307 of the terminal 17, a series of common messages such as “Whatwas the date, time, year?”, “Were there any witnesses to the event?”, or“What is his name, address, employer?” can be pretyped and listed in amessage stack format in the window 307. The associate attorney need onlyselect the desired pretyped message from the stack and select the sendcommand. Similarly, potential objection messages can be added for quickaccess to the common message stack in the communication window of theassociate defending attorney's terminal 18. The common message stack ateach terminal can be customized by editing, adding or deleting messagestherefrom.

A time command is also available for the command windows of the attorneyterminals.

Executing the time command enables the attorney to identify delay timesbetween the end of a specific question and the beginning of thewitness's answer. Similarly, using the time command, witness andopposing attorney conferral time can be recorded. Underlying thiscommand, a system timing routine operates to log event timing in aseparate data file which the time command can access for specificrequests.

The system timing routine also operates as an alarm clock for schedulinga deposition. Referring back to FIGS. 7 a and 7 b, using the alarm con,hand while preparing a hierarchical outline for the deposition, theattorney may allocate a projected amount of time for each category orsubcategory. The projections can be monitored by the system timingroutine, and indicate to the attorney through a system communicationthat projections are not being held. Specifically, while under aparticular category, the system timing routine begins to count down thetime allocated to the subcategory as described above. If the attorneycompletes the category before time runs out, the extra time will beadded to a buffer. If instead time runs out, the alarm clock sends acommunication to the attorney indicating that the projected time hasexpired and indicates the buffer status. The buffer is periodicallyupdated by subtracting the time which lapses beyond the projected value.If after receiving the system communication the attorney remains in acategory past the projected time period, the alarm clock sends a newsystem communication every thirty (30) minutes (as a default value).

As the deposition progresses, whenever the attorney enters thetree-structured outline mode, the time value displayed for eachcategory, as illustrated in FIG. 7 b, reflects the time remaining ineach specific category. The time value for the category currentlyselected (and its parents in the tree-structure) continually updates soas to display the scheduler's minute by minute count down. Reschedulingor other time value changes are automatically reflected throughout theentire tree-structured display, enabling better time management of thedeposition.

When a confidential restriction is placed on specific subject matter ofa deposition, the examining attorney often forgets to remove therestriction when the subject has changed. By selecting the alarm clockcommand and then the subsequently listed confidentiality wake-upcommand, the terminal 15 automatically sends a system communicationevery fifteen minutes (can be changed through a set up screen) to remindthe examining attorney to clear the restriction from the transcript ifpossible. In addition, upon selecting a different category orsubcategory in the hierarchical outline, a system communication is alsoimmediately transmitted as a reminder. The confidentiality con, landalso provides a mechanism for disabling and enabling local networkcommunication from reaching off-site client terminals, unlessspecifically directed thereto.

In one embodiment of the present invention, translated sentences arecommunicated from CAT system 13 via serial communication alongcommunication link 21 to terminal 15 and communication link 23 toterminal 17. Communications between terminals 15 and 17 utilize the twoindependent communication links 21 and 23. In another embodiment, thecommunication links 21 and 23 are interconnected and function as asingle communication link (bus) using a TCP-IP protocol on an ethernetcoaxial cable link using an ethernet board in each terminal. Twoethernet boards are desirable in the reporter terminal so that isolationcan be maintained between the opposing sides. A software package“Berkeley Sockets”, developed at the University of California, Berkeley,is the preferred software base for maintaining the communication flowalong the link. Packages implementing “Berkeley Sockets” are availablethrough various third party vendors, including for example FTP Systems,Inc., Purlington, Mass. The communication links 20 and 22 areinterconnected and function in a similar manner via the second ethernetboard in the reporter terminal. Other hard-wired or wirelesscommunication may also be implemented.

Two issues of isolation must be addressed by the communication links inthe present invention. First, the court reporter stores all of thekeystrokes pressed and all of the transcribed alphabetic and numerictext in the CAT system 13. These files must not be modifiable by anyterminal on any down-line communication link. Such modifications wouldcorrupt these files and the transcript would not be certifiable.Therefore, the communication out of CAT system 13 should only be thatnecessary to transmit copies of the stored files. More importantly,communication into CAT system 13 should only be that necessary torequest transmission. Although additional communication features intoand out of CAT system 13 may be added, they are not preferred.

Second, isolation must be established between the defending andexamining terminals. Under no circumstances can the messages between theexamining terminals be intercepted by the defending terminals. Thisrequires that the communication links must be isolated. Isolation may beaccomplished by using an independent hard-wired communication link as isillustrated in FIG. 1 via a serial or coaxial link, for example.Encryption of the communication exchanged may be used as a safeguard.However, if a wireless link is used, encryption becomes a requirement.Although various encryption schemes are contemplated, a preferred schemeinvolves the use of direct-sequence spread spectrum transmission. Suchtransceivers are relatively inexpensive and, by their very nature,provide encryption. Other exemplary communication links which might beimplemented include token ring, parallel, infrared, RF (radiofrequency), etc.

FIG. 9 illustrates additional features of the present invention whichthe court reporter and attorneys may utilize. As stated previously, thecourt reporter usually tapes the entire deposition or trial as a back upto the stenographic recording, and uses the audio tape to help intranslating the proceedings. To avoid clumsy and time consumingsearching through the audio tape to locate Q & A's, a synchronizationscheme has been added to electronically associate each stenographicallyrecorded Q & A with the actual position on the audio tape of the audioof the Q & A. With synchronization, the court reporter can quickly andeasily locate audio on the tape.

Specifically, a tape recorder 351 is attached via a communication link353 to the reporter terminal 13. Upon receiving an entire Q & A, thereporter terminal 13 requests and receives a position indication fromthe tape recorder 351. The marker position is generated by the taperecorder by a tape length counter 355 and associated circuitry. Thereporter terminal 13 stores and associates each such position indicationwith the corresponding keystrokes. Thereafter, when necessary, the courtreporter can display the position indicator and quickly locate thedesired audio manually. Alternatively, communication link 353 can beused by reporter terminal 13 to communicate control signals to the taperecorder 351 so as to automatically position and play-back previouslyrecorded audio upon request.

Instead of receiving position indications from the tape recorder 351,the reporter terminal 13 may send an audio signal representing anupcoming Q & A number to recorder 351 for recording. This audio signalmay be either a voice-synthesized indication of the number count, aidinga manual search, or a digital data stream that is detectable by therecorder 351 using low or high speed searching directed via terminal 13.

Similarly, a video camera 361 may also be used to video tape ofproceedings. Although video camera 361 may be able to independentlystore and play audio tapes, a separate video recorder, video recorder363, is illustrated as being connected via a communication link 365 forthis purpose. As with the audio recorder, video recorder 367 can be usedin a synchronization scheme for associating and accessing video eventswith the transcribed text through communication link 369. A televisionmonitor 371 provides for playing back recorded video information.

If the video or audio system is fully automated through the control ofreporter terminal 13, upon a request from a judge or attorney to readback a Q & A, the court reporter may play back the associated audio andvideo by merely locating the requested Q & A on reporter terminal 13,possibly through a lexical search, and selecting a play-back command.

Also contemplated is the use object-oriented association of recordingsusing video frame grabbers and sound grabbers resident in reporterterminal 13 which digitally sample, store, associate and synchronize thevideo and audio of a proceeding. Synchronization would take place usinga direct association of the transcribed text in a common programmingobject with the audio and video frames.

Using position synchronization for locating and playing audio and videoalso proves to be a valuable tool for the attorney. If the attorneydesires to play portions of the audio or the video, for example, duringtrial, the attorney may use the lexical searching capabilities of theattorney terminals to locate the desired Q & A's, and may thenautomatically play the associated audio or video back for the judge andjury.

Referring to FIG. 10, a court room scenario is shown using the presentinvention. A first chair, examining attorney 401 holds a portablecomputer tablet 403 which has a wireless transceiver 405 attachedthereto. The tablet 403 may provide pen-based or touch sensitive inputand acts as a direct substitute to the attorney's paper note pad. In oneembodiment, the wireless communication link is established using RF(radio frequency), spread-spectrum transmissions for the inherent dataencryption associated therewith. As stated previously, other types ofRF, ultrasonic or infrared transmission and data encryption schemesmight also be used.

To guarantee isolation of the court reporter's transcription files, theterminal 13 only manages outgoing communication and requests thereforeas described previously in relation to the configuration of FIG. 1. Asexplained above, although such communication exchanges involving thereporter terminal 13 do not require encryption, secure communicationsbetween the attorney terminals must exist to prevent interception by theopposing side (not shown). As illustrated, secure communication isaccomplished between the terminals 15 and 17 using the independenthard-wired communication link 23. Encryption using a pseudo-random,secret spreading code common only to transceivers 405 and 409implementing direct-sequence spread spectrum transmission secures thecommunication link between the attorney terminal 403 and either of theterminals 15 and 17. Other encryption schemes may also be used.

Wireless transmission may also be used by additional wirelesstransceivers (not shown) for use by additional on-site attorneysstaffing the lawsuit. A communication line 411 may be used to establisha communication link with additional terminals used by attorneys,client's, or experts who are not in the court room, i.e. are remotelylocated off-site. Although the communication line 411 is a telephoneline attaching to an internal modem of terminal 17, other communicationmethods such as a cellular RF link might be used. The communication line411 provides for the active participation of off-site persons who mightotherwise be unable, because of cost or time, to attend.

The local attorney terminals 403, 15, and 17 can control the flow ofdata through communication line 411. For example, if confidentialinformation is being described, the attorney terminal 15 can blocktransmission off-site to a client by disabling outgoing communicationvia the off-site link via a command on the control line 259.

Together, the transceivers 407, 405, and 409 and the links 23 and 411form a single communication link between the terminals. Channelisolation exists between the examining attorneys. File isolation existsat the CAT terminal 13. The characteristics of this single, overallcommunication link is transparent to the attorneys operating theterminals, they need only choose a destination and can send messages.Although not shown, the opposing attorneys also possess a similarcommunication system on a second, single communication link.

More specifically, the first chair, attorney 401, uses portable tablet403 to record notes, receive translated Q & A's and exchange messageswith the second chair at terminal 15. The first chair can either directthe message to one of the supporting attorneys at a terminal or globallysend messages to all available attorneys at all of the terminals on thelink. Similarly, messages can be received from such terminals. Althoughnot necessary, a communication pathway hierarchy is preferred whereinall messages directed between the first chair and lower numbered chairsare intercepted by the second chair terminal for screening andredirection. In this way, the second chair attorney prevents the firstchair attorney from being distracted by unnecessary, unimportant, ormisdirected messages. Additional levels of hierarchy between the secondand lower numbered chairs may also be implemented.

A terminal such as portable tablet 403 or attorney terminal 15 mightalso be used by the judge during and after trial for reviewing andanalyzing the lawsuit. When facing a ruling on the matter, the judgemight also use the features described above to determine whether properevidentiary form was achieved in the Q & A. Similarly, for review andanalysis, the jury might be provided such terminates during trial ordeliberation to help reach their decisions. Especially beneficial ineither situation would be the ability for audio and/or video review asdescribed in relation to FIG. 9 above.

FIG. 11 illustrates an alternate embodiment wherein, instead ofresponding indirectly to stenographic keystroke translation, thereporter terminal 13 directly performs voice transcription andassociation. Added to the reporter terminal 13, a microphone 451generates an analog voice signal representative of the words spokenduring a proceeding. The analog voice signal passes along acommunication link 453 to an A/D (analog to digital) converter boardhoused within reporter terminal 13. This A/D converter samples anddigitally converts the analog voice signal to a series of digital valuesrepresentative of the words spoken at the proceeding. Linear predictivecoding (“LPC”) techniques are then used either remotely (through anoff-site computer link) or locally (within reporter terminal 13) totranscribe the deposition.

Additionally, the reporter terminal 13 may also use the stenographickeystroke input as a basis for training the voice transcription system.Specifically, the reporter terminal 13 breaks the digital voice signalinto segments, and, using LPC techniques, synchronizes throughassociation groups of the segments with corresponding keystrokes fromthe stenographic recorder 11. The process of associating the segmenteddigital voice signal with the corresponding stenographic keystrokesprovides a method for training the voice transcription of the reporterterminal 13. Once trained, voice transcription may more effectivelysupersede the stenographic recorder 11 in generating a writtentranscript of the proceeding.

FIG. 12 is used to illustrate two alternate embodiments where the courtreporter is not linked to the transcription network. In a firstembodiment, voice transcription is directly managed by an attorneyterminal 555 using a microphone 551 via a line 559. Voice transcriptioncan occur either locally, within terminal 555, or remotely via acommunication link 565. Additionally, as described previously,transcription, association, synchronization, and message exchanges wouldoccur between terminal 555 and an associates terminal 563.

A second embodiment is illustrated wherein terminals 555 and 563 providefor message exchange and message association with the correspondingrecorded audio on the tape, transcription not being available. Messageassociation would occur using by either retrieving and storing theposition indication from recorder 553, or sending for recording, audioor digital signals for manual of automated play back. A more detaileddescription of the functionality of such message association andplay-back can be found above.

FIG. 13 is a perspective view of another embodiment of the presentinvention. Instead of using a keyboard, the attorney terminals 15 and 17might adopt pen-based or touch-sensitive screen input. Pens 601 and 603and the screens 251 and 303 provide an attorney a substitute for a padand Post-it® brand notes. The associate attorney may open a computerizedrepresentation of the Post-it® brand note (a “computer note”), scribblea message thereon using the pen 603, and press the send command on thecommand line 309. Unlike actual Post-it® brand notes, computer notes canbe expanded. Each expanded computer note is displayed in the defaultsize yet has an associated scroll-bar and expand and reduce buttons thatcan be manipulated so that an oversized message can be fully read.

The terminal 17 directly sends the entire bit-mapped image of thecomputer note to the attorney terminal 15 for display in a tiled format(cascading may also be selected as a default) on the screen 251. Theattorney operating the terminal 15 may then modify, delete, move, etc.,the received computer note using the pen 601 and the commands from thecommand line 259. Message exchanges in the opposite direction and notetaking, as previously described, works in a similar manner. Moreover,instead of sending a bit-map of the computer note, handwritingrecognition software may be used to convert the note to a text format tomake the notes more legible to the recipient. The recognition softwaremay be selected directly through command line 259 or through a set-updefault.

The transcription network of FIG. 13 is illustrated to operate withouttranscription input; however, if present, transcribed text can also bedisplayed by opening transcription windows as described above.

As with the previous embodiments, the associates terminal 17 has theability to display and manipulate the messages displayed on the attorneyterminal 15. For example, the associate can: 1) delete computer notesthat have been used or become stale; 2) replace incomplete orunrecognizable notes; 3) rearrange the tiled or cascaded notes to bettercommunicate priority or to organize subject matter; or 4) modify thesignificance indicator causing message border colors to change or flash(described more fully above) on the attorney terminal 15 indicatingmessage priority.

FIG. 14 illustrates a flow-chart describing the communication protocolused by the attorney terminals for communicating with the reporterterminal (or CAT system) 21. Ordinarily, at the completion of atranscription of a full Q&A, the reporter terminal 13 transmits the Q&Adown-line to along communication links 21 and 20, for example. If theattorney terminals, such as the terminals 15, 17, 16 and 18, arelistening, they receive and update their screen displays appropriately.A specific protocol has be implemented to accommodate: 1) requests bythe attorney, A specific pro preliminary translations; 2) attorneyterminals which have not “heard” all of the Q&A transmissions; and 3)screening of Q&A's by a senior chaired attorney to those down-line suchas remote, client terminals.

Specifically, as represented by a begin block 651, upon connecting tothe communication link with the reporter terminal, the attorneyterminals initiate the communication protocol. At a block 653, theattorney terminals construct a log-in packet, a packet of bytes ofdigital information, which consists of: 1) a source identifier,identifying itself as the source of the communication; 2) a destinationidentifier, identifying the reporter terminal as the destination; and 3)a pointer indicating the attorney terminal's current position, i.e., thepointer identifies the last Q&A that the attorney terminal has receivedfrom the reporter terminal.

The pointer to the last or most recent Q&A is referred to hereafter as a“self.current” pointer. If an attorney terminal is logging in for thefirst time in a deposition, the value of the “self.current” pointer isequal to zero (0) to indicate that no Q&A's have yet been received.Likewise, if the attorney terminal is logging in a second or subsequenttime (because of either an accidental or purposeful disconnection), the“self.current” pointer is set to the last Q&A number received prior tothe disconnection.

As represented by a block 655, the attorney terminal sends the log-inpacket to the reporter terminal, and waits for a response. Uponreceiving the log-in packet, the reporter terminal enters the sourceidentifier into a log-in list, stores the “self.current” pointer, andtransmits a responding packet containing a source identifier,destination identifier, and the number of the most recently translatedQ&A. The most recent Q&A number is maintained by the reporter terminal's“rptr.current” pointer.

Upon receipt of the response packet, as illustrated by a block 657, theattorney terminal first determines the number of outstanding packetsneeded. This is done by subtracting the attorney terminal's“self.current” pointer value from the received, reporter terminal's“rptr.current” pointer value. The resulting difference (the number ofneeded Q&A's) is then compared to a threshold value. If the differenceis less than the threshold value (a “yes” to the decision at block 657),the attorney terminal builds an update request packet, containingsource, destination end update indicators, and transmits the packet asrepresented by blocks 661 and 663, respectively. In this instance, theupdate request packet is very similar to the log-in packet,containing: 1) a source identifier; 2) destination identifier; and 3)pointer to the attorney terminal's current position.

If the difference is greater than the threshold value (a “no” at block657), the attorney terminal prompts the attorney using the attorneyterminal for the range of Q&A's that are actually necessary, asrepresented by a block 659. The range is selected by choosing theearliest Q&A desired. The range would then be from the earliest Q&A tothe most current Q&A as determined by the reporter terminal. Theprompting for a range enables the attorney to evaluate the number ofoutstanding Q&A's that have not been received, and request only thosethat are truly needed. The thresholding prevents large deposition filedownloading to late-coming attorneys where unnecessary. Thereafter, atthe block 661, the attorney terminal constructs an update request packetusing the selected range, and transmits it to the reporter terminal atthe block 663. In this instance, the update request packet contains: 1)a source identifier; 2) destination identifier; and 3) the number of theearliest Q&A desired.

At block 663, after transmitting the update request, the attorneyterminal awaits an update. The reporter terminal responds by sending therequested range. However, if the requesting attorney terminal or ahigher chaired attorney has placed a Q, A or Q&A restriction upon therequesting terminal for all or part of the requested range of Q&A's, thereporter terminal transmits substituted text, such as “RestrictedViewing!” instead of the actual text translation where necessary. Thereporter terminal tracks restrictions in the log-in table. Similarly,whenever a Q&A translation is complete, the reporter terminal onlytransmits the text of the translation to those attorney terminals havingno restrictions placed thereon. To those attorney terminals withrestrictions placed thereon, the substitute text is transmitted.

Upon receiving the requested Q&A's or substitute text therefor, asrepresented by a block 665, the attorney terminal stores the requestedQ&A's and updates its “self.current” pointer. At this point, theattorney terminal is considered up to date. Thereafter, bothperiodically and upon an attorney's direct request (in response tospecific keyboard or mouse input), the attorney terminal constructs anupdate packet, at a block 667, and repeats the cycle beginning withblock 655. This looping cycle ensures that the attorney terminal staysup to date, even if the attorney terminal for any reason temporarilystops monitoring the communication link during the deposition.

Whenever a request packet is received which contains a “self.current”pointer which is equal to the reporter terminal's “rptr.current”pointer, the current Q&A pointed to by the “rptr.current” pointer isresent along with any partial translation for the next Q&A that has beencompleted. The current Q&A is always resent to make sure that theattorney terminal possesses the full translation thereof (in case only apartial translation is currently stored). The transmission of thepartial translation of the subsequent Q&A allows the specificallyrequesting attorney to display the ongoing translation.

The value of the threshold depends on nature of the communicationnetwork and user preference. If the network is sluggish, the thresholdshould be set lower to off-load competing communication on the channel.To accommodate each user's preference, control of the threshold value isbe placed at each attorney terminal.

In addition, although the thresholding decision is made by the attorneyterminal as represented by the block 657, in an alternate embodiment,the reporter terminal makes the decision. Specifically, referring toFIG. 15, a flow-chart of an alternate embodiment of the softwareprotocol used by the reporter terminal 13 is shown. Upon receipt ofeither a log-in or update request packet, the reporter terminalinitiates its response as represented by a begin block 681. At a block683, the source indicator is compared to the log-in list to determinewhether the currently requesting attorney terminal has been loggedtherein. If not, the source indicator (or source identifier) is added tothe log-in list.

At a decision block 685, the reporter terminal compares the received“self.current” pointer with its “rptr.current” pointer. If thedifference between the two is less than a threshold value (as indicatedby the “yes” path), the reporter terminal: 1) checks for viewingrestrictions in the requested range, at a block 691; 2) constructs aresponse which includes all of the requested Q&A's with substituted textwhere restricted, at a block 693; 3) transmits the response to therequesting attorney terminal, at the block 695; and 4) exits the routineat the end block 697. However, if the difference at the decision block685 is greater than the threshold value (indicated by the “no” pathway),prior to encountering blocks 691, 693, 695 and 697, the reporterterminal first prepares and sends a range packet inquiry to therequesting terminal and awaits an update response therefrom,respectively at blocks 687 and 689. The range inquiry packet indicatesthe total range of outstanding Q&A's. The attorney terminal responds byprompting the user for the earliest desired Q&A, constructs anotherupdate request packet, and transmits it to the reporter terminal whichawaits its receipt at the block 689. From this responsive update requestpacket, the reporter terminal completes the protocol through theremaining blocks as described above.

Additionally, it is obvious that the embodiments of the presentinvention described hereinabove are merely illustrative and that othermodifications and adaptations may be made without departing from thescope of the appended claims.

1. A transcription system, comprising: a transcription module configuredto generate one or more transcription signals that include transcribedtext representing one or more spoken words in real time, wherein thetranscription module synchronizes the transcribed text in the one ormore transcription signals with one or more recordings that correspondto the transcribed text; a first communication device communicativelycoupled to the transcription module over one or more communicationlinks, wherein the first communication device is configured to: displaythe transcribed text in the one or more transcription signals inresponse to receiving the one or more transcription signals from thetranscription module over the one or more communication links; define amessage that annotates a selected portion of the transcribed textdisplayed on the first communication device; and transmit the messagethat annotates the selected portion of the transcribed text displayed onthe first communication device over the one or more communication links;and a second communication device communicatively coupled to thetranscription module and the first communication device over the one ormore communication links, wherein the second communication device isconfigured to: receive the message transmitted from the firstcommunication device over the one or more communication links; displaythe selected portion of the transcribed text annotated with the messagetransmitted from the first communication device, wherein the secondcommunication device displays the selected portion of the transcribedtext with the message annotating the selected portion of the transcribedtext in response to receiving a selection of the message on the secondcommunication device; and play a portion of the one or more recordingssynchronized with the selected portion of the transcribed text in theone or more transcription signals.
 2. The transcription system of claim1, wherein the message that the first communication device defines toannotate the selected portion of the transcribed text includes one ormore pretyped messages selected on the first communication device from aset of pretyped messages accessed on the first communication device. 3.The transcription system of claim 1, wherein the second communicationdevice displays the message transmitted from the first communicationdevice in a communications stack in response to receiving the messagefrom the first communication device, wherein the communications stackincludes the message and one or more additional messages displayed in anorder based on a priority level that the first communication devicedefines for the message.
 4. The transcription system of claim 1, whereinthe second communication device is further configured to: automaticallyperform a real time evaluation of the one or more transcription signalsgenerated by the transcription module; and selectively indicate anexistence of a potential legal objection to the one or more spoken wordsrepresented in the transcribed text in response to the real timeevaluation of the one or more transcription signals.
 5. Thetranscription system of claim 1, wherein one or more of the firstcommunication device or the second communication device are furtherconfigured to prevent a remote communication device furthercommunicatively coupled to the one or more communication links fromreceiving the one or more transcription signals in response to the firstcommunication device or the second communication device placing aconfidential restriction on the transcribed text in the one or moretranscription signals.
 6. The transcription system of claim 1, furthercomprising a third communication device configured to: intercept themessage that the first communication device transmits to the secondcommunication device over the one or more communication links, whereinthe one or more communication links provide a hierarchical communicationpathway that enables the third communication device to intercept themessage transmitted from the first communication device prior to thesecond communication device receiving the message; and redirect themessage intercepted from the first communication device to the secondcommunication device over the one or more communication links, whereinthe second communication device receives the message transmitted fromthe first communication device in response to the third communicationdevice redirecting the intercepted message to the second communicationdevice.
 7. The transcription system of claim 1, wherein the firstcommunication device controls, over the one or more communication links,the display of the message and the selected portion of the transcribedtext annotated with the message on the second communication device. 8.The transcription system of claim 1, wherein the one or morecommunication links securely isolate the message transmitted from thefirst communication device to the second communication device from oneor more additional communication devices further communicatively coupledto the one or more communication links.
 9. The transcription system ofclaim 2, wherein the one or more pretyped messages selected on the firstcommunication device to annotate the selected portion of the transcribedtext represents an observed demeanor of a witness associated with theone or more spoken words.
 10. The transcription system of claim 1,wherein the first communication device and the second communicationdevice are further configured to store the message and the selectedportion of the transcribed text annotated with the message to enablesubsequent review of the message and the selected portion of thetranscribed text annotated with the message.
 11. The transcriptionsystem of claim 1, wherein to synchronize the transcribed text in theone or more transcription signals with the one or more recordings thatcorrespond to the transcribed text, the transcription module is furtherconfigured to: associates the transcribed text in the one or moretranscription signals with a position indicator generated from the oneor more recordings corresponding to the transcribed text; communicate asignal associated with the one or more spoken words to a recorder thatcaptures the one or more recordings corresponding to the transcribedtext; and associate a common object-oriented programming object with thetranscribed text in the one or more transcription signals and the one ormore recordings corresponding to the transcribed text.
 12. Thetranscription system of claim 8, further comprising a reporter terminalthat includes the transcription module, a first Ethernet board for thefirst communication device and the second communication device, and asecond Ethernet board for the one or more additional communicationdevices, wherein the reporter terminal uses the first Ethernet board tocommunicate the one or more transcription signals to the firstcommunication device and the second communication device in isolationfrom the one or more additional communication devices.
 13. Thetranscription system of claim 8, wherein the one or more communicationlinks encrypt the message transmitted from the first communicationdevice with one or more secrets shared between the first communicationdevice and the second communication device in response to the firstcommunication device wirelessly transmitting the message over the one ormore communication links, thereby securely isolating the message fromthe one or more additional communication devices.
 14. The transcriptionsystem of claim 8, wherein the one or more communication links includean independent hard-wired link between the first communication deviceand the second communication device, whereby the first communicationdevice transmitting the message to the second communication device overthe independent hard-wired link securely isolates the message from theone or more additional communication devices.
 15. A transcriptionmethod, comprising: generating, with a transcription module, one or moretranscription signals that include transcribed text representing one ormore spoken words in real time; synchronizing the transcribed text inthe one or more transcription signals with one or more recordings thatcorrespond to the transcribed text; displaying the transcribed text inthe one or more transcription signals on a first communication devicecommunicatively coupled to the transcription module over one or morecommunication links, wherein the first communication device defines amessage that annotates a selected portion of the transcribed text;transmitting the message that annotates the selected portion of thetranscribed text from the first communication device to a secondcommunication device over the one or more communication links, whereinthe second communication device displays the selected portion of thetranscribed text with the message annotating the selected portion of thetranscribed text in response to receiving a selection of the message onthe second communication device; and playing a portion of the one ormore recordings synchronized with the selected portion of thetranscribed text in the one or more transcription signals.
 16. Thetranscription method of claim 15, wherein the message that the firstcommunication device defines to annotate the selected portion of thetranscribed text includes one or more pretyped messages selected on thefirst communication device from a set of pretyped messages accessed onthe first communication device, and wherein the one or more pretypedmessages selected on the first communication device to annotate theselected portion of the transcribed text represents an observed demeanorof a witness associated with the one or more spoken words.
 17. Thetranscription method of claim 15, further comprising displaying themessage transmitted from the first communication device in acommunications stack on the second communication device, wherein thecommunications stack includes the message and one or more additionalmessages displayed in an order based on a priority level that the firstcommunication device defines for the message.
 18. The transcriptionmethod of claim 15, further comprising indicating a potential legalobjection to the one or more spoken words represented in the transcribedtext in response to the second communication device performing a realtime evaluation of the one or more transcription signals.
 19. Thetranscription method of claim 15, further comprising preventing a remotecommunication device further communicatively coupled to the one or morecommunication links from receiving the one or more transcription signalsin response to the first communication device or the secondcommunication device placing a confidential restriction on thetranscribed text in the one or more transcription signals.
 20. Thetranscription method of claim 15, further comprising: intercepting themessage that the first communication device transmits to the secondcommunication device over the one or more communication links at a thirdcommunication device, wherein the one or more communication linksprovide a hierarchical communication pathway that enables the thirdcommunication device to intercept the message transmitted from the firstcommunication device prior to the second communication device receivingthe message; and redirecting the message that the third communicationdevice intercepted from the first communication device to the secondcommunication device over the one or more communication links, whereinthe second communication device receives the message transmitted fromthe first communication device in response to the third communicationdevice redirecting the intercepted message to the second communicationdevice.
 21. The transcription method of claim 15, further comprisingcontrolling the second communication device displaying the message andthe selected portion of the transcribed text annotated with the messageat the first communication device, wherein the first communicationdevice deletes the message on the second communication device over theone or more communication links to control displaying the message andthe selected portion of the transcribed text annotated with the messageon the second communication device.
 22. The transcription method ofclaim 15, further comprising securely isolating the message transmittedfrom the first communication device to the second communication devicefrom one or more additional communication devices furthercommunicatively coupled to the one or more communication links, whereinsecurely isolating the message from the one or more additionalcommunication devices includes: communicating the one or moretranscription signals from a reporter terminal that includes thetranscription module to the first communication device and the secondcommunication device using a first Ethernet board for the firstcommunication device and the second communication device, wherein thereporter terminal further includes a second Ethernet board for the oneor more additional communication devices to communicate the one or moretranscription signals using the first Ethernet board in isolation fromthe one or more additional communication devices; encrypting the messagetransmitted from the first communication device with one or more secretsshared between the first communication device and the secondcommunication device in response to the first communication devicewirelessly transmitting the message over the one or more communicationlinks, thereby securely isolating the message from the one or moreadditional communication devices; and transmitting the message to thesecond communication device over an independent hard-wired link that theone or more communication links include to securely isolate the messagefrom the one or more additional communication devices.
 23. Thetranscription method of claim 15, further comprising storing the messageand the selected portion of the transcribed text annotated with themessage on the first communication device and the second communicationdevice to enable subsequent review of the message and the selectedportion of the transcribed text annotated with the message.
 24. Thetranscription method of claim 15, wherein synchronizing the transcribedtext in the one or more transcription signals with the one or morerecordings corresponding to the transcribed text includes: associatingthe transcribed text in the one or more transcription signals with aposition indicator generated from the one or more recordingscorresponding to the transcribed text; communicating a signal associatedwith the one or more spoken words represented in the transcribed text toa recorder that captures the one or more recordings corresponding to thetranscribed text; and associating a common object-oriented programmingobject with the transcribed text in the one or more transcriptionsignals and the one or more recordings corresponding to the transcribedtext.